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PMP  Advisor  es 


Produced  whenever  a  part,  material, 
or  process  experiences  an  unusual 
failure  or  problem  that  can  be  shared 
with  the  community  to  help  avoid 
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Distribution  Restriction  Statement 


Distribution  of  this  ikicumenl  » limited  to  United  States  Government  agencies  and  their  eontractecs  who  require  know  ledge  of  its 
contents  to  aid  them  m  minimizing  adverse  effects  on  MDA  and  Federal  Government  programs  This  information  has  been  c  rut  pi  led 
and  presented  as  accurately,  completely,  and  objectively  as  posublc  consistent  with  the  primary  objective  of  alerting  potentially 
affected  programs  as  early  as  possible  To  the  best  of  MDA  knowledge,  this  Advisory  docs  not  conlam  any  untrue  statement  of 
fact,  or  omit  a  statement  of  fact  necessary  in  order  to  make  the  statements  made  (in  light  of  the  circumstances  under  which  the 
statements  w  ere  made)  rat  misleading  A  MDA  Advisory  does  not  reflect,  nor  should  it  be  interpreted,  to  imply  a  negative  judgment 

informal  ion  mat’  be  altered  revised  or  rescinded  by  subsequent  developments  or  additional  tests:  these  changes  could  be 
communicated  by  other  MPA  documents  Neither  MDA.  the  United  Stales  Government,  nor  any  person  acting  on  Us  behalf  assumes 
any  liability  resulting  from  any  distribution  or  use  of  (his  information 

This  document  may  contain  information  that  is  export-controlled  under  either  the  International  Traffic  in  Arms  Regulations  (ITAR). 
22  CFR  120- 1 30.  or  the  Export  Adnmistrabon  Regulations  (BAR),  15  CFR  730-774  Accordingly,  it  is  not  to  be  disclosed  to 
foreign  nationals,  either  in  the  US.  or  abroad,  without  the  approval  of  MDADXSS  nor  without  appropriate  export  authorization.  if 
required  such  os  an  expert  license,  a  technical  assistance  agreement,  or  an  ITAR  exemption  letter  Violations  of  these  regulations 
[je  '  bnj  j  ■!  n  a  n Jge . . 

Summary: 

MPA  has  experienced  various  issues  with  real-time  embedded  operating  sy  stems  (OS).  Embedded  OS  tools 
are  ty  pically  used  in  processor  based  commercial  products  to  shorten  development  time  and  provide  proven 
and  tested  industry  standard  interfaces  ami  controls  to  (he  product  in  the  smallest  memory  space  Real-time 
systems  also  allow  for  predictable  and  controllable  system  response  times  It  is  important  for  the  users  of 
embedded  software  products  to  thoroughly  test  the  product  to  verily  the  embedded  code  behaves  as  it 
should,  and  no  undcsirod  operations  or  effects  can  be  triggered.  Users  must  also  keep  the  embedded  code 
upduted  with  all  revisions  of  software  and  any  patches  that  may  be  olTcred  by  the  embedded  software 
company  to  fix  various  problems  Failure  to  do  so  can  often  lead  to  operational  problems  w  ith  the  product 

Examples: 

A  problem  was  identified  with  flie  embedded  software  within  a  network  module  used  in  MDA  Sensors 
hardware  The  fault  appears  isolated  to  flic  Wind  River  operating  system,  indicated  by  a  resource  access  bit 
that  does  not  get  cleared  properly .  A  firmware  upgrade  is  being  installed  to  the  network  module  and  all 
spares,  which  incorporates  a  change  from  Wind  River  to  prevent  a  resource  contention  issue.  This  change  is 
necessary  to  prevent  future  deadlocks.  The  problem  caused  the  system  to  require  a  hard  reset,  which  took 
.30  minutes 

Similarly  ,  an  MDA  Target  experienced  a  bootup  anomaly  in  circuit  cards  using  the  W  ind  River  Tornado  OS 
kernel  Investigation  into  flic  problem  traced  the  issue  to  a  cache  bit  that  was  not  clewed  properly  after 
execution  of  a  subroutine.  The  problem  caused  the  card  to  “hang”,  requiring  a  reset. 
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Counterfeits 


Counterfeit  parts  found  in  several 
programs,  and  has  implemented  an 
agency-wide  process  (e.g.  MDA 
Policy  Memo  #50)  to  combat  the 
problem 

Not  just  Integrated  Circuits,  but 
diodes,  transistors,  chip 
capacitors,  and  COTS  hardware 
items  have  been  found  to  be 
counterfeit 

Most  practical  first  step  is  to  beef 
up  procurement  practices  to  avoid 
likely  sources  and  increase 
awareness  to  detect  them  at 
incoming  inspection 
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Watch  where  you  buy! 


^rld  of  Component  Sot^^ 


DO  NOT 

PROCURE  FROM 

NON-APPROVED 
SOURCES!! 


Reality  of  Independent  Distributors 


z — 


Website  photo 


Actual  “facility” 


Both  distributors  sell  (sold)  to  defense  contractors.  Both 
advertise  availability  to  thousands  of  components.  Both  are 
residential  businesses. 


Website  address 


Actual  “facility” 
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Part  Failures 
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No  matter  how  hard  you  try,  part  failures 
happen,  often  because  of  seemingly 
innocuous  reasons 


•  Chip  capacitor  failures  because  the  sintering  oven 
experienced  a  power  glitch  at  the  factory  (process 
control) 

•  Power  supply  failure  due  to  a  tiny  transformer  breaking 
loose  inside  a  hybrid.  The  epoxy  pre-form  used  to  glue 
it  down  was  found  to  be  contaminated  by  mishandling 
with  silicone 

•  SMA  jacks  with  intermittent  signal  loss  found  caused  by 
a  bad  batch  of  poorly  formed  center  contact  barrels 
making  it  into  finished  product 
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As  Integrated  Circuits  shrink  to  ever-smaller 
geometries,  sensitivity  to  Electrostatic 
Discharge  (ESD)  is  a  continuing  problem 

Pink  Poly  is  not  that  good 

Moisture  sensitivity  on  small  parts  forces 
tighter  humidity  controls  -  too  low  and  ESD 
becomes  catastrophic,  too  high  and  sensitive 
devices  pick  up  too  much  moisture  prior  to 
assembly  on  boards 

Worker  awareness  and  training,  MSD 
tracking  processes  more  important  than  ever 
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FOD 


Sensitive  parts  that  must  be  assembled  in 
clean  areas  demand  a  robust  Foreign  Object 
Damage  (FOD)  program 

Shop  cleanliness: 

-  condition  of  floors,  ceilings 

•  We  have  found  crumbling  ceilings,  broken  floor  tiles, 
falling  insulation  in  FOD-sensitive  assembly  areas 

-  clutter,  wire  clippings,  solder  splashes 

-  tool  condition 

•  Flaking  chrome  plate,  rust,  tape  residue  on  scissors 

-  hand  lotion  (silicone  contamination) 

•  Sorry  ladies,  but  most  hand  lotion  has  various  silicones 
(often  Dimethicone)  in  it 
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Defective  Titanium 


Critical  materials  are  occasionally  a  tempting 
thing  to  take  shortcuts  to  save  money  or 
increase  profit 

Titanium  billet  supplier  was  skipping  a 
couple  process  steps  to  reduce  cost  on  billet 
(an  intermediate  form  of  metal) 

-  Was  not  rolling  it  to  spec  size,  was  cutting  it 

-  Was  not  performing  spec  heat  treatment 

Resulted  in  potentially  reduced  material 
strength  of  titanium  going  into  making 
aerospace  parts 
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Design  Flubs 


A  lanyard  pull  connector  failed  due  to  the 
way  the  lanyard  was  oriented  and  the  length 
of  the  lanyard 

-  Resulted  in  pulling  a  connector  off  the 
hardware 

Shut-off  switches  poorly  positioned,  too 
close  to  where  operators  were  resting  their 
hands 

-  Equipment  accidentally  shut  off  during  test 

Need  to  carefully  review  design  choices  to 
make  sure  designs  perform  as  intended 
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Materials  Issues 

that  won’t  j  ust  go  away... 


Tin  Whiskers 

-  Pure  tin  plating,  plus  Zinc  and  Cadmium  plating,  will 
whisker  given  enough  time  -  awareness  and  mitigation 

Red  Plague 

-  Silver  plated  wire  will  be  destroyed  by  red  plague  (copper 
galvanic  corrosion)  given  time,  moisture,  and  plating 
damage 

Silicone  and  Loctite  Migration 

-  Silicone  lubricants  never  cure,  and  creep  and  migrate  to 
virtually  any  surface  -  avoid  use  where  it  can  cause 
issues 

-  Loctite  (thread  lock  compound)  does  not  cure,  and  creeps 
to  unwanted  areas  -  wipe  off  excess 

Gold  Embrittlement  -  solder  will  become  brittle  if  mixed 

with  gold  from  gold  plated  pads 

-  Remove  95%  of  gold  from  pads  prior  to  soldering  by 
tin/remove  process 
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Embedded  Software 


Embedded  operating  systems  and  firmware 
often  has  bugs  and  glitches  that  even  the 
originating  supplier  doesn’t  know  about  until 
users  discover  them 

Real-time  OS  vendor  patches  for  a  popular 
OS  product  number  into  the  dozens 

-  Make  sure  you  implement  all  appropriate 
patches  and  thoroughly  test  the  deployable 
code  in  the  product 

A  GPS  vendor  had  a  glitch  in  firmware  that 
caused  the  time  code  to  skip  -  once  every 
few  days 
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Qual  ty  is  Deceiving 


Watch  for  references  to  MIL-SPEC  quality, 
where  vendor  is  only  performing  some  of  the 
steps  for  QPL,  skipping  others 


Actual 
Optocoupler 
vendor  datasheet 


Harnessing  the  Power  of  Technology  for  the  Warfighter 


Bonding  involves  several  steps  besides  applying 
the  glue  and  putting  pieces  together 

-  Need  proper  cleaning,  surface  prep,  and 
testing  to  assure  success 

Soldering  still  a  major  source  of  problems 

-  Watch  for  hidden  solder  joints  and  implement 
proper  inspection  techniques  (x  ray,  etc) 

-  Proper  hole  fill,  tip  temperatures,  wave  mm  m  m 

exposure  time,  solder  composition,  gold 

removal  (gold  embrittlement), . the  list  goes 

on . 

Carefully  design,  execute,  and  monitor  all  critical 
processes 
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Summary 


Carefully  select,  implement,  execute,  and  test 
all  processes 

Be  wary  of  where  and  from  whom  you 
procure  -  there  is  an  ocean  of  unscrupulous 
operators  out  there 

-  Cultivate  a  deeper  relationship  with  suppliers, 
and  share  company  concerns,  provide 
oversight  and  flow-down  of  knowledge  and 
requirements 

Spend  the  extra  time  and  money  to  sample 
test  and  check  critical  items.  It  pays 
dividends  by  avoiding  costly  issues  later 
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